New Understanding of the Bethe Approximation and the Replica Method

In this thesis, new generalizations of the Bethe approximation and new understanding of the replica method are proposed. The Bethe approximation is an efficient approximation for graphical models, which gives an asymptotically accurate estimate of the partition function for many graphical models. The Bethe approximation explains the well-known message passing algorithm, belief propagation, which is exact for tree graphical models. It is also known that the cluster variational method gives the generalized Bethe approximation, called the Kikuchi approximation, yielding the generalized belief propagation. In the thesis, a new series of generalization of the Bethe approximation is proposed, which is named the asymptotic Bethe approximation. The asymptotic Bethe approximation is derived from the characterization of the Bethe free energy using graph covers, which was recently obtained by Vontobel. The asymptotic Bethe approximation can be expressed in terms of the edge zeta function by using Watanabe and Fukumizu’s result about the Hessian of the Bethe entropy. The asymptotic Bethe approximation is confirmed to be better than the conventional Bethe approximation on some conditions. For this purpose, Chertkov and Chernyak’s loop calculus formula is employed, which shows that the error of the Bethe approximation can be expressed as a sum of weights corresponding to generalized loops, and generalized for non-binary finite alphabets by using concepts of information geometry. The replica method is a method invented in statistical physics for analyzing typical behaviors of random statistical models. Although the replica method is non-rigorous, it gives empirically correct results for various problems. However, many involved techniques employed in the replica method prevent study and understanding of it. The contribution of the second part of the thesis is regarding clarification of the replica method. The main tool for the purpose is the method of types, which is a well-known elementary tool in information theory. From the method of types, clear derivation and interpretation of the replica method are obtained. As a consequence, it is revealed that the replica method gives the same results as the cavity method, and that the replica method on the replica symmetry assumption implies the validity of Bethe approximation.